Buyer Beware: Lessons 
Learned from edTPA 
Implementation in 
New York State 


by Deborah Greenblatt and Kate E. O'Hara 

A s states across the country continue their implemen¬ 
tation of the edTPA, a complex and high-stakes 
certification requirement for teacher certification, 
there are important lessons for educators and education advocates to learn from 
New York State’s implementation. As Linda Darling-Hammond, developer and 
promoter of the edTPA, cautioned at the 2014 American Educational Research 
Association meeting: “New York is a prototype of how not [original emphasis] to 
implement teacher performance assessment.” 1 

edTPA stands for the Teacher Performance Assessment Portfolio, an assess¬ 
ment of teacher readiness developed by The Stanford Center for Assessment, 

Learning and Equity (SCALE) and the American Association of Colleges for 
Teacher Education (AACTE) but nationally distributed and scored by Pearson 
Education, Inc. It differs from previous assessments in that it purports to measure 
“performance” by requiring student teachers to compile a portfolio, including les¬ 
son plans, student work samples, a short classroom video (15 to 20 minutes), and 
a lengthy “instructional commentary” of 40 to 60 pages. 
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Currently, there are 622 educator preparation programs in 35 states and the 
District of Columbia participating in edTPA. Some states are still exploring its 
use while others require edTPA as part of program completion or for state licen¬ 
sure. 2 Among them, New York's story is unique: Although the New York State 
Education Department had begun working with Pearson in 2009 on its own 
teacher performance assessment, it switched to the edTPA when it became avail¬ 
able in February 2012. The handbooks and rubrics were made available to faculty 
and students in New York's schools of education that same spring. 3 New York 
only conducted one year of field testing before fully implementing the edTPA as 


It is our hope that educators , activists, and policy 
makers will join our ejfortfor a certification process 
that represents what is important to the profession. 


a high-stakes assessment. 4 

As a result of the rapid rollout, faculty at colleges of education had little time 
to reflect on their data and prepare their students for success: “We have basically 
set up a cohort of our students to fail,'' warned Jamie Dangler, vice president of the 
United University Professions (UUP), the union of State University of New York 
educators, to New York State Education Department officials in January 2014, 
“and the consequences will be disastrous for students and teaching programs.''' 

With the federal push to standardize a national evaluation requirement for 
pre-service teachers, all states and their educators must also consider and contend 
with the impact of profit-oriented corporations in the teacher preparation process. 
The certification of teachers has been taken out of the hands of the states and now 
turned over to a for-profit company that has much to gain from a national adop¬ 
tion of the edTPA: Pearson Education, Inc. 

Concerns over the corporatization of teacher certification are fueled by 
Pearson's lack of transparency. When participating in Pearson workshops, train¬ 
ings, or test scoring, faculty have to sign non-disclosure agreements. Faculty are not 
allowed to share materials with their colleagues or their students. 5 Furthermore, 
although the edTPA Myths and Facts document asserts the criteria for selecting and 
training scorers is “rigorous,” the teacher candidates' score reports do not include 
the qualifications of their scorer nor is specific data about edTPA current scorers 
readily available online. 

It is our hope that educators, activists, and policy makers will benefit from the 
lessons we have learned in New York and join our effort for a certification process 
that does not standardize teacher education programs but rather draws upon an 
effectively designed certification process and represents what is important to the 
profession, not politicians and corporations. 
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Lesson One: edTPA is Called a Teacher Performance Assessment—But 
that Doesn’t Mean it is One 

While many stakeholders would agree that a performance assessment is a 
more effective way to measure teacher readiness than a pencil-and-paper test, 
the edTPA cannot fill that role. The edTPA relies greatly on teacher candidates’ 
reading, writing, and technological skills. Candidates are allowed to include only 
up 15 to 20 minutes of video to “feature the teaching and learning emphasis” for 
their subject area. 6 

For first-year teachers to be “effective,” the edTPA weighs heavily on data 


A professor colleague elaborates , “We used to share 
successes and challenges in seminar...Now all we seem 
to do is go through the [edTPA] handbook. ” 


analysis skills, while de-emphasizing skills such as adaptability, relating to stu¬ 
dents’ interests, and fostering a cooperative environment. Additionally, the 
lengthy and tightly structured edTPA requirements have changed the focus of 
the student teaching experience and seminar from preparing for the first year of 
teaching to preparing to pass a test and create lessons under constraints that make 
the test an unauthentic assessment. 7 

Student teachers are faced with the challenge of manipulating both the struc¬ 
ture and the content of their lessons to meet the demands of the edTPA. While 
student teachers are typically stressed about their coursework or teaching lessons, 
they now have the additional anxiety of making sure their teaching practices are 
labeled and organized according to the limitations of the edTPA questions, often 
struggling to manipulate what has been genuinely successful, into something that 
instead meets the unrealistic and unfamiliar demands of the test. A professor col¬ 
league elaborates, “We used to share successes and challenges in seminar, working 
together, digging deep. Now all we seem to do is go through the handbook page 
by page to make sure we understand what is expected of our students.” 

Lesson Two: The edTPA Privileges Certain Student Teaching Placements 

The challenges of the edTPA are exacerbated in schools in low-income com¬ 
munities where our K-12 students often are not scoring well on standardized 
tests. Not only are these schools more likely to have scripted curricula, but they 
also have students with a variety of special needs. 8 Teacher candidates will need 
special permission to deviate from the mandated curriculum to showcase their best 
work in their videos, which can be a challenge depending on several factors such as 
how much principal surveillance the cooperating teachers are under or how much 
cooperating teachers are willing to “break the rules.” These conflicts often cause 
cooperating teachers to decide not to host a student teacher in future semesters. 9 
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Although the edTPA purports to assess how general education teacher can¬ 
didates address the needs of special education students as per their Individualized 
Education Program goals and the needs of English language learners, these 
prompts are not relevant in certain settings. Candidates placed in more homog¬ 
enous settings will not face these additional demands, while those in more 
diverse classrooms must clearly address the needs of these students to score well. 10 
Although these teacher candidates may be better prepared for their careers when 
they student teach in such settings, schools of education may be motivated to 
sacrifice that preparation to improve edTPA passing rates. 


There is no evidence the edTPA has predictive 
validity> the ability to forecastfuture “success, ” 
for any measure. 


Lesson Three: The edTPA Scoring is Inconsistent 

Although the edTPA is based on National Board for Professional Teaching 
Standards, there is no evidence the edTPA has predictive validity, the ability 
to forecast future “success,” for any measure. 11 SCALE, the originators of the 
edTPA, made a statement on the validity and reliability of the edTPA as follows: 

A set of validation studies was conducted to confirm the content validity, job 
relevance, and construct validity of the assessments. In combination, these studies 
documented that the assessment is well-aligned to the professional standards it 
seeks to measure, reflects the actual work of teaching, and that the score measures 
a primary characteristic of effective teaching. Inter-rater reliability was evaluated 
using several different statistical tests. edTPA reliabilities reported here range from 
.83 to .92 (indicating the percentage of scorer agreement). 12 


This statement, however, addresses reliability or validity when it comes to 
inter-rater reliability only. In the same document, SCALE states the edTPA gives 
states the “ability to use a nationally available common measure that is valid and 
reliable to evaluate pre-service teachers' readiness to teach.” 13 It is certainly argu¬ 
able if this common measure is truly valid and reliable. The National Center for 
Teacher Quality agrees saying: 

What may be a very good culminating exercise for any program to administer 
is not necessarily a sufficiently valid and reliable measure of either an individual 
teacher or the quality of a program. For example, the edTPA allows candidates to 
choose the lessons they will deliver, rehearse as many times as they wish, and edit 
the videotape of their teaching. If a prospective elementary teacher chooses to teach 
a lesson on parallel lines rather than on equivalent fractions (because she really 
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dislikes fractions), and even then edits out an instructional faux pas, is the resulting 
lesson a valid assessment of her overall teaching skills? 14 

Additionally, many teacher candidates and teacher educators question the 
consistency in the scoring of the test. A colleague from a State University of New 
York campus shares her experience: “My students did pass, but ironically the 
ones that weren't as strong — you know, in pedagogy, or in classroom manage¬ 
ment— they passed, and passed with higher scores than my stronger candidates.” 
Experiences such as this call into question the reliability of the evaluations and the 


“Ironically, my students that weren't as strong — 
in pedagogy, or in classroom management — they 
passed, and passed with higher scores." 


training of the Pearson evaluators. 

According to a former Pearson evaluator, originally when portfolios were 
reviewed at scoring centers, two people graded each of the portfolios. However, 
scoring is now done remotely and scorers are recruited from across the country, 
even if their state does not use the edTPA. Also, now only one person scores each 
edTPA portfolio. 15 Although there are “quality control” measures put in place by 
Pearson, with random portfolios being “back read” by a supervisor, it is not clear 
how often back reading is done. 16 Given that scorers are paid per portfolio and are 
not held accountable until one of their portfolios is randomly selected for addi¬ 
tional scoring, there is no existing measurement of inter-rater reliability. 

According to Nancy A. De Korp, coordinator of Education Programs, Office 
of Higher Education for the NYSED, there is also a protocol if the total score is 
at, or around, a passing score. In this case, a second scorer will review the portfolio. 

A third scorer (a scoring supervisor) will evaluate the portfolio if either 1) 
Scorer 1 and Scorer 2 are discrepant (more than one score point apart) on any 
rubric, or 2) Scorer 1 and Scorer 2 are on opposite sides of the recommended 
professional performance standard (for decision consistency). A .5 score occurs 
when two scorers have evaluated a portfolio and their score on the rubric is aver¬ 
aged. Scorer 1 and Scorer 2 may have exact agreement or adjacent agreement on 
each rubric. Any discrepant rubrics (more than one score point apart) are sent to a 
supervisor for resolution. 17 

However, we have seen instances where teacher educators, those who have 
assessed teacher candidates' performance multiple times across the semester, have 
a different evaluation than the Pearson scorers. One teacher candidate shared 
with us that, although she was the only person in her student teaching seminar 
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to get an A, she failed the edTPA. Her professor evaluated the portfolio herself, 
using the Pearson rubrics, and disagreed significantly with the scorer s findings. 
As a result she is considering an appeal; however she would have to pay $200 to 
do so. 18 

Allowing per diem scorers of the edTPA to be the gatekeepers to the profes¬ 
sion depersonalizes the relationship between teacher candidates and their students, 
cooperating teachers, field supervisors, and professors in a very troubling way. 19 


Allowing per diem scorers of the edTPA to be the 
gatekeepers to the profession depersonalizes the 
relationship between teacher candidates and students. 


Lesson Four: The edTPA Shifts the Focus of the Student Teaching experi¬ 
ence to Test Preparation 

Teacher candidates spend hours reading, rereading, examining, and under¬ 
standing their edTPA content specific handbook, the Making Good Choices guide¬ 
book, and other associated edTPA documents. In addition to the time devoted to 
typical planning, there are rubrics and prompts that must be dissected, discussed, 
and understood in order to earn a passing edTPA score. Candidates spend many 
additional hours analyzing student data, compressing and uploading videos, and 
writing pages of commentary, leaving little time for planning of high quality les¬ 
sons. Jen Boerner, a graduate student at SUNY Brockport, said that the biggest 
drawback to the edTPA was the lack of attention she was able to pay to all of her 
special education students: “‘I feel I lost out on a lot of student teaching. I really 
couldn’t do as much as I wanted. I couldn’t go over all the lesson plans I wanted to 
try out because I was teaching to the test. That was unfortunate.’” 20 In some cases, 
instead of teaching follow-up lessons in the days after their edTPA “learning seg¬ 
ment,” candidates spent their nights answering the writing prompts for Pearson. 
It’s also important to note that in addition to the time committed to edTPA 
and student teaching, many teacher candidates are taking additional courses to 
maintain their financial aid eligibility. With so much at stake, the reality for many 
teacher candidates is that they often need to leave their coursework unfinished, 
miss classes and deadlines, or simply hand in acceptable, rather than exemplary 
work so they can focus on constructing their edTPA portfolio. 21 

Lesson Five: The edTPA Privileges Candidates and Institutes of Certain 
Financial Status 

One must consider the repercussions of the increase in the cost of the exams 
package, which has now doubled to over $600. The exams package includes the 
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edTPA plus the Educating All Students exam, Content Specialty Tests, and the 
Academic Literacy Skills test. Although Pell grant recipients are eligible to get a 
voucher for the $300 edTPA portion of the exam, they are not guaranteed one. 
Only 600 edTPA vouchers were distributed across all of New York State for all 
eligible teacher candidates. 22 

For example, when Hobart and William Smith Colleges applied for vouchers 
for its 24 students who received federal Pell grants, they received an e-mail from 
ESTestVoucher @Pearson.com saying that vouchers were allocated proportion¬ 
ally to institutions based on the number of undergraduate Pell recipients reported 


Teacher candidates who can afford to attend 
well-funded teacher credentialingprograms often 

enjoy additional support services. 


by each institution. For Hobart and William Smith's 24 eligible students, they 
received only one voucher. 23 At New York University, dozens of undergraduates 
and graduate students with specific financial aid and GPA requirements were eli¬ 
gible to enter a lottery to try to “win” one of only eight vouchers. 24 Unfortunately, 
not all schools advertise these vouchers either. In some cases, students don't know 
that such a voucher exists unless an individual faculty member makes an effort to 
inform their students. 

Teacher candidates who can afford to attend well-funded teacher credentialing 
programs often enjoy additional support services, including the services of a full¬ 
time edTPA resource person. 25 Schools with funding for that kind of position can 
provide edTPA workshops, seminars, and one-on-one consultation. Because of 
the benefits of these funded positions, teacher candidates' course work or seminars 
do not have to be dedicated to edTPA preparation. Some colleges have subject 
specific edTPA coordinators who conduct workshops for students on the edTPA 
throughout their teacher education program which then allows teacher candidates 
to prepare for portions of their portfolio prior to student teaching experience. On 
the other hand, in colleges where funding is limited, online modules are often 
used or a website hosting edTPA resources is created to reach a broader audience 
of teacher candidates across disciplines. However, this approach leaves candidates 
with more general information and no human interaction. Additionally, while a 
single edTPA coordinator at one college might be in charge of all edTPA content 
portfolios, their counterparts at a wealthier school might have coordinators for 
each certification area. 

Lesson Six: The edTPA Privileges Candidates from Certain Linguistic and 
Cultural Backgrounds 

A teaching candidate may have carefully planned and successfully taught an 
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effective “learning segment” but unless the candidate has also learned the language 
of the edTPA exam, followed particular directives, understood rubric objectives, 
and crafted their commentary by thoroughly reflecting the terminology designed 
and used by SCALE, she or he risks a failing score. This goal of learning the 
language of a teacher “bar exam” specifically disadvantages teacher candidates of 
color, as noted by Christine D. Clayton, Department chair of the Pace University 
School of Education: 

Recent data from other states (where the assessment is not a certification 


“/ wonder if the edTPA will systematically reproduce 
race and class inequalities , like every other 
high-stakes standardized test. ” 


exam) indicate that some groups, including teaching candidates of color and those 
from linguistic minority groups, were failing edTPA at disproportionate rates. 
Other reports showed evidence of candidates who performed well on edTPA, but 
did not earn fair supervision reports when actually teaching students. 26 

Wayne Au, associate professor in the education program at the University of 
Washington, expresses a legitimate concern, “Given the severe lack of teachers of 
color and teachers from working-class backgrounds, I wonder if the edTPA will 
systematically reproduce race and class inequalities, like every other high-stakes 
standardized test.” 27 

Lesson Seven: The edTPA Technology Requirements Privilege Certain 
Candidates and Institutions 

The edTPA requirements assume that institutions have the technology that 
candidates need to complete their portfolio; it is also assumed that candidates 
possess the technology skills needed to create and share their edTPA portfolio. 
Purchasing or renting the required recording device for the taping of learning 
segments creates a financial burden on many candidates and institutions of higher 
education. Furthermore, the use of technology can prove challenging even to those 
with basic familiarity with the equipment. Teacher candidates must have the digi¬ 
tal literacy skills to not only know how to use the device appropriately but they 
must also know how to “compress” and upload their videos according the required 
Pearson specifications. There are many issues related to the video component 
of the edTPA portfolio, from the quality of the recording to technical problems 
during the videotaping process. 28 One City University of New York teacher can¬ 
didate contacted her supervisor about her learning segment. She explained that 
even though the lesson she taught went well and she pleased with the work her 
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students had produced, she was not able use a video clip from that particular lesson 
because her camera ran out of battery and her backup recording device ran out of 
hard drive space. 

Other candidates reported technical issues in the uploading of their portfolios 
to the edTPA platform. 29 Students have reported it taking them approximately 
two hours to review all the prompts, submit all the sections, and to upload their 
materials to meet the specific criteria, which include everything from the size and 
style of the font to the specifications for compressing and uploading videos. 


“The moral of this story is to predict what the 
raters might want, and give it to them , no matter 
how relentlessly repetitive and monotonous...” 

BUYER BEWARE 

The practical and ethical implications for implementing the edTPA are 
complex and significant. From our experiences in New York State, it is arguable 
whether or not the edTPA adequately assesses teacher performance. However, 
what we can say with certainty is that the edTPA privileges student teacher 
placements; shifts student teaching of candidates to test prep by candidates; has 
inherent inconsistencies in the scoring by Pearson; privileges certain candidates 
and higher education institutions; and makes assumptions about candidates' tech¬ 
nology access and skills. 

As teacher educators, we have learned significant lessons, and so have our 
teacher candidates. “The moral of this story is to predict what the raters might 
want, and give it to them, no matter how relentlessly repetitive and monotonous 
the rubrics may be.” 30 A follow-up lesson is that the teacher candidates do truly 
“perform” on this test, determined to create a show that their audience will like. 

With edTPA portfolios being outsourced nationally, teacher candidates can only 
hope that their performance earns them applause from the lone worker being paid 
$75 per portfolio. Although currently the edTPA is being used by more than 70 
percent of teacher certification programs in the country, the flaws are evident. A 
word of advice from New York: Buyer beware. GD 
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